Lesson 7
Timothy Luna
8/23/2019
Edge Detection: 2D Operators
Intro
Let’s take what we’ve learned in one dimension and apply it to a two dimensional image.
Remember how edges are related to gradients; we developed some operators that you can apply to gradients and we discussed first applying filters to smooth out the noise and in what order that can be done to save operations.
Derivative of Gaussian Filter 2D
In 2D it’s not just a derivative but also the direction of the derivative.
Suppose we want to take the derivative in the x direction.
\[(I\otimes g) \otimes h_x = I \otimes (g \otimes h_x)\]
What this sub x is implying is that we are only applying the derivative in the x direction. This could be the Sobel operator or any other operator that requires a derivative. It’s a small filter taking a derivative. g is the gaussian function and the associative property allows for moving the operation around.
This gives us the derivative of the gaussian smoothed image from before. It’s preferable to do it this way, but why?
Quiz: Smoothing
Why is it preferable to apply \(h\) to the smoothing function \(g\) and apply the result to the image?
Is the horse dead yet? The smoothing function \(g\) is generally smaller than the image. Applying \(h\) to the smaller function requires fewer operations that then don’t need to be repeated.
Effect of \(\sigma\) on Derivatives
Smoothed Gaussian
How big of a gaussian should we use? What effect does a changing spread have on the derivative? As your \(\sigma\) grows your smoothing effect will magnify; the same effect holds true when computing derivatives, enhancing the magnitude of the derivative as it grows in space.
What effect does this have in practice? Smaller values will capture finer features as edges and larger values will filter out to only large features.
Canny Edge Operator
How do we actually find the edges?
- Smoothing derivatives to suppress noise and compute the gradient.
- Threshold the output to find regions of ‘significant’ gradient.
- ‘Thin’ the results to get localized edge pixels. (Will talk more about this later; this reduces thick edges to thin contour.)
This operation, the Canny operator, works as follows:
- Filters with the derivative of a Gaussian.
- Find the magnitude and orientation of the gradient.
- Perform non-maximum suppression where you thin multi-pixel wide ‘ridges’ down to a single pixels width. (Thinning)
- Linking and thresholding. Define two thresholds, a high and a low, and use the high threshold to start edge curves and the low threshold to continue them.
In python this is called with
import cv2
im = cv2.imread('image')
cv2.Canny(im,...)Canny Edge Detector
So, let’s walk through Canny edge detection in practice.
Step 1: Image
Given some image, we would like to detect edges using the method that Canny created.
Step 2: Gradient
Although this looks like edges to a person, it’s simply the magnitude of the gradient. It needs to be worked on prior to being useful.
Step 3: Thresholding
Getting rid of anything in the image that is above / below a threshold allows us to quickly and easily discard things that most likely are not edges. Lena got a haircut…
Step 4: Thinning
Thinning allows representing large and wide bands of ‘edge’ as thin and simple contour lines by checking for local maxima along the gradient direction.
Along the direction of the gradient, as seen in the images to the right, there will be a local maxima somewhere on the line of the gradient direction. The ‘thinned’ edge is represented solely by that local maxima. This can require subpixel accuracy when interpolating!
Step 5: Canny Threshold Hysteresis 1
- Apply a high threshold to detect strong pixels.
- Link those strong edge pixels to form strong edges.
- Apply a low threshold to find weak but plausible edge pixels.
- Extend the strong edges to follow weak edge pixels.
This assumes that noisy edges that are close to strong edges are likely edges.
For Your Eyes Only Demo
This code block reads in the frizzy and froomer images, uses a Canny filter to compute the edges in the image, and then uses a boolean operation to detect where edges in the two images overlap.
import cv2
# Helper function to show images.
from cs6476helpers import imshow
# Read in the data
frizzy = cv2.imread("frizzy.png")
froomer = cv2.imread("froomer.png")
# Show off the original images
imshow(frizzy)
imshow(froomer)
# Calculate the edges using the Canny filter
# Note that 10 and 20 for thresholds here are arbitrary.
frizzy_edges = cv2.Canny(frizzy, 10, 20)
froomer_edges = cv2.Canny(froomer, 10, 20)
# Show off the edges
imshow(frizzy_edges)
imshow(froomer_edges)
# Overlay the edges
shared_edges = frizzy_edges & froomer_edges
# And show off where they overlap!
imshow(shared_edges)Frizzy
Edges
Froomer
Edges
Shared Edges
Quiz: For Your Eyes Only
What is the secret code in the image above? (cue James Bond music…)
Canny Results
It’s really hard to know when an edge image is good. What will you use them for?
The Canny operator tends to be better at pulling out edges you want to use for future processing.
What size of kernel should we use in a Canny filter?
Large \(\sigma\) detects large edges and small detects small.
Quiz: Canny Edge
Is the Canny operator sensitive to noise? This is mostly false and is dependent on the \(\sigma\) chosen for the smoothing operation. The smaller the \(\sigma\) the more likely the image will be to pick up noise.
Single 2D Edge Detection Filter
The 1D case of the second derivative of the Gaussian was an ‘inverted Mexican hat’ operator. The ‘zero crossings’ of the second derivative correspond to the edges.
There’s a problem! How does this extend to two dimensions? If you’re calculating second partial derivatives then you have \(f_{xx}\), \(f_{xy}\), \(f_{yy}\), and \(f_{yx}\). Which do you use?
Some of them; Use the Laplacian.
\[ \nabla ^ 2 h = \frac{ \partial ^ 2 f }{ \partial x ^ 2 } + \frac{ \partial ^ 2 f }{ \partial y ^ 2 } \]
This is a single operator that can find edges on both the X and the Y. Yay! Do the Mexican sombrero dance.
Edge Demo
import cv2
# Helper functions
from cs6476helpers import imshow
from cs6476helpers import surf
from cs6476helpers import norm
from cs6476helpers import LoG
# Load in the image and display it
img = cv2.imread("lena.png")
imshow(img,title="Lena")
# Make the image into a grayscale image and display it
img2 = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
imshow(img2,title="Boring Lena")
# Make a Gaussian filter and plot as surface
filter_size = 10
filter_sigma = 3
gaussian = cv2.getGaussianKernel(filter_size,filter_sigma)
gaussian = gaussian * gaussian.T
surf(gaussian)
# Apply the filter and show the smoothed image
lena_smoothed = cv2.filter2D(img2,-1,gaussian)
imshow(lena_smoothed,title="Blurry Lena")
# Method one, shift differential. Shift the image to L, then R, then show the difference.
lena_left = np.roll(lena_smoothed,-4,1)
lena_right = np.roll(lena_smoothed,4,1)
imshow(norm(lena_left*1.+lena_right*1.))
# Method two, Canny filter on both the unsmoothed and smoothed images
canny_lena = cv2.Canny(img2, 60, 110)
imshow(canny_lena)
uncanny_lena = cv2.Canny(lena_smoothed, 60, 110)
imshow(uncanny_lena)
# Method three, Laplacian of Gaussian
laplacian = LoG(4, 1.)
surf(laplacian)
laplacian_edges = cv2.filter2D(img2,-1,laplacian)
imshow(norm(laplacian_edges),title="Laplacian Lena")
Original Lena
Monochrome Lena
Gaussian Surface
Fuzzy Lena
